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ABSTRACT 

The  Aeronautical  Design  Standard  33  (ADS-33)  prescribes  a  set  of  manoeuvres  and  limits  for 
assessing  the  handling  qualities  of  military  rotorcraft.  In  developing  an  ADS-33  maritime 
addendum,  the  Aircraft  Maintenance  and  Flight  Trial  Unit  (AMAFTU)  and  Defence  Science  and 
Technology  Organisation  (DSTO)  have  collaboratively  worked  towards  the  definition  for  a 
maritime  hover  manoeuvre.  Initial  testing  to  validate  this  proposed  manoeuvre  in  the  DSTO  Air 
Operations  Simulation  Centre  (AOSC)  flight  simulator  has  proved  problematic.  Although  pilots 
could  successfully  complete  a  land-based  hover  within  performance  limits  in  the  flight  simulator, 
they  experienced  difficulty  in  maritime  conditions.  Pilots  were  unable  to  perceive  small  distance 
changes  in  the  AOSC  simulated  maritime  environment  in  some  conditions. 
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Distance  Discrimination  Thresholds  During  Flight 
Simulation  in  a  Maritime  Environment 

Executive  Summary 


The  Aeronautical  Design  Standard  33  (ADS-33)  prescribes  a  set  of  manoeuvres  and  limits  for 
assessing  the  handling  qualities  of  military  rotorcraft.  In  developing  an  ADS-33  maritime 
addendum,  the  Aircraft  Maintenance  and  Flight  Trial  Unit  (AMAFTU)  and  Defence 
Science  and  Technology  Organisation  (DSTO)  have  collaboratively  worked  towards  the 
definition  for  a  maritime  hover  manoeuvre. 

Initial  testing  to  validate  this  proposed  maritime  hover  manoeuvre  in  the  DSTO  Air 
Operations  Simulation  Centre  (AOSC)  flight  simulator  has  proved  problematic.  Although 
pilots  could  perform  a  land-based  hover  in  the  flight  simulator,  they  experienced  difficulty 
in  maritime  conditions,  far  exceeding  the  usual  degradation  in  performance  observed 
when  shifting  from  actual  flight  trial  to  simulated  flight  trial.  In  accordance  with  the 
proposed  Maritime  Hover  performance  limits  (Appendix  A),  pilots  were  expected  to 
maintain  plan  position  within  3  ft  of  the  hover's  origin,  whilst  also  maintaining  heading 
and  altitude.  This  was  to  be  done  with  reference  to  a  buoy  50  m  away.  In  the  DSTO  AOSC 
Maritime  Hover  trial,  plan  position  could  not  be  maintained  under  any  conditions. 

It  was  suggested  that  perceptual  limitations  were  the  reason  for  poor  performance  in  the 
DSTO  AOSC  Maritime  Hover  trial.  A  short  experiment  was  required  to  test  whether  or  not 
a  change  of  3  ft  between  the  aircraft  and  the  buoy,  at  distance  of  50  m  and  at  a  constant 
altitude,  was  large  enough  to  be  detected.  The  experiment  described  in  this  paper  was 
conducted  for  this  purpose.  Its  aim  was  to  determine  the  minimum  distance  the  aircraft 
must  move  from  its  original  position  for  a  change  in  position  to  be  perceived.  This 
minimum  distance  was  defined  as  the  distance  discrimination  threshold. 

For  both  high  and  low  sea  states,  the  thresholds  were  found  to  be  larger  than  the  3  ft 
'desirable'  performance  limit.  If  a  pilot  is  required  to  maintain  plan  position  within  a  limit 
that  is  too  small  to  be  detected  at  a  perceptual  level,  it  will  only  be  by  chance  than  the 
hover  is  successfully  completed.  This  is  not  adequate  assurance  for  testing  a  potential 
manoeuvre  in  the  simulator. 

To  rectify  the  problems  pilots  experience  when  performing  the  hover  in  the  simulator,  3 
solutions  may  be  employed.  These  are  as  follows: 

•  Improve  simulator  technology  to  include  more  of  the  fine  visual  cues  a  pilot 
expects. 
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•  Specify  how  far  from  the  buoy  the  pilot  should  hover.  A  3  ft  movement 
backwards  or  forwards  will  change  the  image  projected  onto  the  screen  by  a 
larger  proportion  at  closer  distances,  decreasing  the  threshold  for  discrimination. 

•  Review  the  maritime  hover  performance  limits  (for  plan  position  in  particular) 
and  widen  them  to  account  for  the  degraded  performance  observed  in  the 
simulator. 

A  combination  of  these  options  may  prove  to  be  the  most  effective  solution. 

The  results  of  this  experiment  suggest  that  the  thresholds  for  distance  discrimination  are 
situated  very  close  to  or  beyond  the  3  ft  'desirable'  limit  of  the  maritime  hover  task.  This  is 
true  in  the  absence  of  any  task  other  than  identifying  distance.  It  is  therefore  unreasonable 
to  expect  pilots  to  maintain  the  current  'desirable'  and  sometimes  'adequate'  distances 
within  the  AOSC  simulator.  While  simulator  improvement  continues,  a  review  of  these 
limits,  or  alternatively  a  prescription  for  the  precise  distance  to  hover  from  the  buoy, 
should  be  considered.  This  will  ensure  that  limits  for  the  maritime  hover  are  a  true 
reflection  of  the  aircraft's  ability  to  perform  the  manoeuvre  and  that  such  an  ability  is  not 
impeded  by  a  pilot's  inability  to  perceive  small  distance  changes  in  the  AOSC  simulated 
maritime  environment. 
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Plan  position 

A  helicopter's  position  in  the  horizontal  plane  (See  Appendix  A). 

Maritime  Hover  performance  limits 

The  limits  prescribed  for  the  successful  completion  of  the  maritime  hover  manoeuvre.  The 
limits  are  demarcated  into  limits  for  'desirable',  'adequate'  and  'inadequate'  performance. 

Db  (Baseline  distance) 

At  baseline,  the  aircraft  sits  50  m  from  the  buoy.  Db  represents  this  baseline  or  central  distance. 
It  does  not  represent  an  actual  location  or  coordinate  of  the  aircraft. 

Dv  (Variation  distance) 

Dv  represents  the  distance  between  the  aircraft  and  the  buoy,  when  that  distance  varies  from 
Db.  It  does  not  represent  an  actual  location  or  coordinate  of  the  aircraft. 

Distance  discrimination  threshold 

The  minimum  distance  the  aircraft  has  to  move  towards  or  away  from  the  buoy  for  that 
movement  to  be  detected. 

Rotor  wash 

The  downwards  blast  of  air  from  the  turning  blades  of  an  operating  rotor  craft.  Over  water  the 
rotor  wash  makes  specific  patterns  that  can  be  of  use  in  judging  altitude  and  plan  position. 
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1.  Introduction 

The  Aeronautical  Design  Standard  33  (ADS-33)  (US  Army  Aviation  and  Missile 
Command,  2000)  prescribes  a  set  of  manoeuvres  and  limits  designed  to  assess  the  handling 
qualities  of  military  rotorcraft,  independent  of  their  size  or  role.  All  current  ADS-33  Mission 
Task  Elements  (MTE)  were  developed  to  be  land-based.  Examples  of  current  ADS-33  MTEs 
include  hover,  pirouette  and  slalom  manoeuvres.  Although  several  maritime  addendums 
have  been  proposed  as  ADS-33  MTEs,  none  at  present  has  been  included.  Previous  work 
completed  by  the  Aircraft  Maintenance  and  Flight  Trials  Unit  (AMAFTU)  and  DSTO  in 
development  of  a  Maritime  Plover  MTE  is  provided  in  DSTO-TN-0936  (Manso  &  Arney, 
2010).  Continuing  this  effort,  a  DSTO  Air  Operations  Simulation  Centre  (AOSC)  trial  was 
undertaken  in  December  2010  to  further  assess  this  proposed  maritime  hover  manoeuvre.  The 
AOSC  simulator  flight  models  are  of  a  sufficient  fidelity  to  allow  for  the  development  and 
assessment  of  MTEs  such  as  the  maritime  hover.  Testing  manoeuvres  in  a  flight  simulator 
such  as  in  the  AOSC  can  reduce  costly  flight  time  and  allow  researchers  to  assess  cognitive 
workload  factors  and  their  impact  upon  task  performance. 

During  the  development  of  the  DSTO  AOSC  Maritime  Hover  trial,  the  maritime  hover 
manoeuvre  required  altitude,  heading  and  plan  position  (see  glossary  for  definition)  be 
maintained  within  certain  prescribed  performance  limits  (Appendix  A;  Table  Al). 
Maintenance  had  to  occur  throughout  a  two  minute  low  altitude  hover  over  open  water,  at  a 
distance  of  approximately  50  m  (Figure  Al),  a  distance  which  was  estimated  by  test  pilots  as 
being  approximately  the  distance  used  during  actual  flight  trials.  In  particular,  a  pilot  was 
required  to  maintain  plan  position  within  3  ft  of  the  hover's  point  of  origin  to  achieve 
performance  levels  within  the  'desirable'  performance  bracket.  The  manoeuvre  therefore 
required,  among  other  skills,  an  ability  to  interpret  small  movements,  of  a  minimum  of  3  ft,  at 
a  distance  of  50  m. 

The  DSTO  AOSC  Maritime  Hover  trial  was  completed  to  confirm  whether  the 
maritime  hover  manoeuvre  could  be  performed,  and  therefore  tested,  in  the  AOSC  simulator. 
The  cockpit  setup  for  the  DSTO  AOSC  Maritime  Hover  trial  is  shown  in  Figure  1.  Pilots  were 
required  to  execute  the  hover  while  maintaining  a  position  50  m  from  a  buoy.  Multiple  sea 
states  (see  Appendix  B,  Table  B1  for  sea  state  information)  were  tested.  In  the  lead-up  to  the 
trial,  preliminary  testing  with  a  helicopter  pilot  was  carried  out  in  the  AOSC  flight  simulator 
to  validate  the  trial's  experimental  parameters,  including  the  hover  distance  of  50  m  and  the 
hover  performance  limits.  The  preliminary  testing  was  problematic.  With  the  DSTO  AOSC 
Maritime  Hover  trial  experimental  parameters,  the  pilot  was  unable  to  achieve  'desirable'  or 
'adequate'  performance  in  the  AOSC  flight  simulator,  in  any  of  altitude,  heading  or  plan 
position.  Plan  position  was  particularly  problematic.  Helicopter  plan  position  was  lost  within 
the  first  30  seconds  of  the  two  minute  interval.  This  was  true  for  all  sea  states  tested. 

The  DSTO  AOSC  Maritime  Hover  trial  compared  hover  performance  in  simulated 
land-based  and  maritime  environments.  The  ground-course  hover  was  generally  completed 
within  'adequate'  performance  limits.  Although  pilot  performance  is  expected  to  degrade 
between  a  flight  trial  and  its  simulator  equivalent,  the  discrepancy  between  performance 
degradation  in  the  simulated  ground-course  and  maritime  hover  manoeuvres  was 
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considerable.  Performance  in  the  simulated  maritime  hover  was  far  worse  than  in  its 
simulated  ground-course  equivalent.  The  maritime  hover  appeared  to  be  far  more  difficult  to 
complete  than  a  land-based  hover.  Feedback  obtained  after  the  trial  showed  that  the  pilots 
experienced  greater  difficulty  maintaining  position  in  higher  sea  states,  as  reflected  in  their 
performance.  The  pilots  also  reported  that  it  was  more  difficult  to  identify  drift  away  from  the 
buoy  than  towards  it. 


Figure  1  Cockpit  position  in  AOSC  partial  dome  flight  simulator  for  the  DSTO  AOSC  Maritime 
Hover  trial 

These  problems  and  discrepancies  presented  a  problem  for  the  outcomes  of  the  DSTO 
AOSC  Maritime  Hover  trial.  Although  the  recommendation  could  have  simply  declared  that 
the  task  was  unachievable  in  the  AOSC  simulator,  more  information  about  why  this  was  the 
case  was  required.  It  was  also  necessary  to  determine  if  and  how  the  task  could  be  altered  (for 
example  with  less  stringent  limits)  so  that  it  could  be  completed. 

During  an  actual  maritime  hover  flight  trial,  a  pilot  will  rely  on  a  host  of  feedback  cues 
to  determine  and  maintain  their  position  over  water.  Some  cues  are  motion  based  -  the 
vestibular  system  detects  self-movement  through  the  rotations  and  accelerations  of  the  head 
during  flight,  providing  important  motion  feedback  cues.  Some  cues  are  visual  -  the  horizon, 
currents  in  the  water,  and  in  particular  the  aircraft's  'rotor  wash'  (see  glossary)  indicate 
change  in  position.  Both  types  are  important  for  adjusting  and  maintaining  position  during 
actual  flight  (Berger,  Terzibas  &  Beykirch,  2007).  Unfortunately,  several  of  these  cues  cannot 
currently  be  replicated  in  the  AOSC  simulator.  The  AOSC  flight  simulator  is  not  motion- 
based,  and  the  image  generator  is  in  constant  development.  Therefore,  only  a  subset  of  visual 
cues  is  available  at  any  give  time.  At  the  time  of  the  DSTO  AOSC  Maritime  Hover  trial  work¬ 
up,  the  most  pertinent  and  useful  of  the  available  cues  were  the  line  of  the  horizon  and  the 
position  of  the  clouds  in  the  sky  and  their  reflections  in  the  water. 
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Without  familiar  visual  cues,  changes  in  distance  between  the  aircraft  and  the  buoy 
may  become  difficult  for  a  pilot  to  evaluate.  This  is  particularly  true  if  a  pilot  relies  heavily  on 
these  cues.  With  only  a  subset  of  the  usual  cues  available,  the  task  of  completing  the  maritime 
hover  in  the  AOSC  flight  simulator  becomes  less  focused  on  a  pilot's  skill  and  technique. 
Instead,  it  becomes  focused  on  their  ability  to  judge  distance  at  a  perceptual  level. 

At  a  distance  of  50  m,  a  3  ft  movement  backwards  or  forwards  in  the  virtual 
environment  changes  the  image  of  the  buoy  projected  onto  the  simulator  screen  by 
approximately  3  mm  top-to-bottom  and  2  mm  left-to-right.  The  buoy  also  moves  in  the  visual 
field  by  a  matter  of  millimetres.  The  magnitude  of  image  size-change  and  movement  may  be 
too  small  to  be  perceived  sitting  2.4  m  from  the  screen  (Figure  1).  The  cues  mentioned  earlier 
complement  the  perceptual  systems  in  actual  flight  trials.  They  indicate  to  the  pilot  that  they 
have  drifted,  even  when  the  size  of  the  image  of  the  buoy  has  not  changed.  Without  these 
cues,  these  additional  indicators  of  movement  are  not  at  hand.  The  pilot's  ability  to  perceive 
distance  change  could  be  compromised.  This  reduction  in  perceptual  ability,  resulting  from 
the  non-availability  of  expected  visual  cues  in  the  AOSC  flight  simulator,  could  be  part  of  the 
explanation  for  the  degradation  in  performance  observed  in  the  simulated  maritime 
environment. 

Fine  perception  is  important  to  the  maritime  hover,  or  indeed,  any  manoeuvre 
requiring  continuous  monitoring  of  slight  movements.  To  maintain  a  fixed  position  while 
hovering,  a  pilot  must  continuously  compensate  for  drift  (Berger,  Terzibas  &  Beykirch,  2007). 

If  a  pilot  is  unable  to  perceive  small  distance  changes,  he/  she  will  be  unable  to  perceive  slight 
amounts  of  drift.  If  drift  is  not  registered,  the  pilot  may  attempt  to  maintain  a  position  that  is 
offset  by  an  error  that  compounds  for  the  duration  of  the  trial.  An  inability  to  perceive  a  3  ft 
movement,  caused  by  a  lack  of  available  cues  in  the  simulator,  will  decrease  the  likelihood 
that  the  manoeuvre  is  successfully  completed. 

It  was  realised  during  the  DSTO  AOSC  Maritime  Flover  trial  that  it  would  be 
beneficial  to  determine  if  perceptual  limitations  were  the  reason  for  poor  performance.  A 
short  experiment  was  required  to  test  whether  or  not  a  change  of  3  ft  between  the  aircraft  and 
the  buoy,  at  distance  of  50  m  and  at  a  constant  altitude,  was  large  enough  to  be  detected.  If  it 
was  not  large  enough  to  be  detected,  some  aspect  of  the  maritime  hover,  the  simulation 
environment  or  the  DSTO  AOSC  Maritime  Flover  trial  experimental  design  had  to  change. 
The  experiment  described  in  this  paper  was  conducted  for  this  purpose. 

Determining  thresholds  for  stimulus  detection  requires  the  application  of  classic 
psychophysical  methods.  A  detection  threshold  refers  to  the  minimum  stimulus  change 
required  for  that  change  to  be  detected.  A  great  deal  of  research  on  detection  thresholds  has 
been  performed  for  stimuli  of  varying  type,  quality  and  strength  (Gescheider,  1997).  Of 
interest  for  the  DSTO  AOSC  Maritime  Hover  trial  is  the  distance  the  aircraft  has  to  move  from 
its  initial  position,  in  the  horizontal  plane,  for  that  movement  to  be  identified. 

As  a  stimulus  moves  towards  or  away  from  the  eye,  the  size  of  its  image  projected 
onto  the  retina  changes.  As  the  stimulus  moves  closer,  the  image  becomes  larger.  Similarly,  as 
it  moves  away,  the  image  becomes  smaller.  However,  in  a  flight  simulator,  the  image  is 
always  the  same  distance  from  the  eye.  Thus,  for  the  DSTO  AOSC  Maritime  Hover  trial,  the 
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detection  threshold  for  movement  towards  or  away  from  the  buoy  concerns  the  detection 
threshold  for  changes  in  the  size  of  the  projected  image  of  the  buoy.  In  an  environment  where 
all  simulated  environmental  stimuli  are  removed,  it  would  be  possible  to  determine  if  changes 
to  the  image  of  the  buoy  of  a  matter  of  millimetres  are  smaller  than  the  threshold  for  size 
detection.  If  changes  of  this  size  are  smaller  than  the  threshold  for  image  size  change 
detection,  they  are  beyond  perceptual  abilities.  Elements  of  the  environment  that  might  affect 
perception  would  have  to  be  removed.  Results  from  such  research  would  be  useful  to 
determine  whether  or  not  the  difficulties  with  the  DSTO  AOSC  Maritime  Hover  trial  are 
purely  perceptual. 

During  the  DSTO  AOSC  Maritime  Hover  trial,  there  could  have  been  factors  other 
than  perceptual  limitations  contributing  to  the  degraded  performance.  Simulated  flight  trials 
are  more  complex  than  pure  detection  threshold  test  environments.  Pilots  could  reliably 
complete  a  simulated  hover  over  land  within  'adequate'  performance  limits,  with  their 
performance  showing  the  amount  of  degradation  to  be  expected  when  shifting  from  actual 
flight  trial  to  simulated  flight  trial.  As  stated  earlier,  a  greater  level  of  degradation  was 
observed  for  the  simulated  maritime  environment.  Differentiating  the  simulated  land 
environment  from  the  simulated  maritime  environment  is  the  availability  and  reliability  of 
visual  cues  and  the  movement  of  the  target  with  wave  motion.  Perceptual  limitations  might 
therefore  be  only  one  of  several  factors  contributing  to  poor  distance  discrimination  in  the 
AOSC  dynamic  simulated  maritime  environment.  Visual  cue  availability  and  target  motion 
could  also  be  contributing  factors. 

Consequently,  the  experiment  described  in  this  paper  was  not  designed  as  a  pure 
detection  threshold  experiment.  Instead,  the  aim  of  the  experiment  was  to  determine  how  far 
an  aircraft  has  to  move  from  its  initial  position,  in  the  presence  of  simulation  environment 
cues,  for  a  change  in  distance  to  be  perceived.  By  establishing  this  threshold  for  distance 
discrimination  in  the  AOSC  simulator,  it  will  be  determined  whether  the  current  maritime 
hover  limits  are  just  very  difficult  to  discern  or  are  in  fact  unobservable.  Furthermore,  a 
recommendation  may  be  made  regarding  the  appropriateness  of  the  'desirable'  and 
'adequate'  limits. 

When  estimating  distance  in  a  simulator  or  during  actual  flight  trials,  given  adequate 
weather  conditions,  certain  visual  cues  will  always  be  useful  (Berger,  Terzibas  &  Beykirch, 
2007).  The  horizon  is  such  a  cue.  The  position  of  a  target  relative  to  the  horizon  changes  as  the 
aircraft  moves  back  and  forth,  and  if  maintaining  a  constant  altitude,  can  indicate  distance  to 
the  target.  Similarly,  the  horizon  indicates  changes  in  pitch  and  roll  as  it  moves  in  the  pilot's 
visual  field  (Berger,  Terzibas  &  Beykirch,  2007).  Equally,  visual  elements  of  the  environment 
such  as  wave  movement  can  hinder  distance  perception.  Yaw  changes  are  difficult  to  gauge  in 
the  maritime  environment  due  to  the  absence  of  distant  objects  (Berger,  Terzibas  &  Beykirch, 
2007).  These  cues  and  distractions  in  the  AOSC  simulator  (horizon  line,  wave  motion,  clouds 
and  cloud  reflections,  lack  of  distant  objects)  will  always  be  present  when  flying  the  maritime 
hover  manoeuvre.  It  is  acknowledged  that  any  threshold  for  distance  detection  that  is 
determined  while  these  cues  are  present  will  be  contingent  upon  these  cues  remaining  in  the 
environment. 
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Based  on  observations  from  the  DSTO  AOSC  Maritime  Hover  trial  and  associated  pilot 
feedback,  the  hypotheses  for  the  experiment  were  as  follows: 

1 .  Participants  would  be  able  to  discriminate  smaller  distances  in  lower  sea  states  due  to 
the  smaller  wave  movement  and  more  static  available  visual  cues. 

2.  Participants  would  be  able  to  discriminate  smaller  distances  for  cases  where  the 
aircraft  moved  towards  the  buoy  rather  than  when  it  moved  away. 

3.  For  both  high  and  low  sea  states,  and  for  both  closer  and  farther  distances,  the  3  ft 
'desirable'  performance  limit  would  be  smaller  than  the  distance  discrimination 
threshold  in  the  AOSC  simulated  maritime  environment,  for  the  DSTO  AOSC 
Maritime  Hover  trial  experimental  parameters. 


2.  Method 


2.1  Participants 

The  participants  for  this  experiment  were  10  male  and  7  female  DSTO  employees 
ranging  from  22  to  39  years  of  age.  The  participants  were  permitted  to  wear  glasses  if 
required. 


2.2  Equipment 

The  DSTO  AOSC  flight  simulator  (predominantly  the  MRH-90  cockpit  and  partial 
dome)  were  the  main  components  used  for  the  experiment.  The  partial  dome  supports  a  6- 
channel  projector  system,  stitching  6  separate  images  to  create  a  200  degree  horizontal  field-of- 
view,  104  degree  vertical  field-of-view,  1200x1600  (per  projector)  resolution  image.  The  AOSC 
comprises  a  fixed  based  simulator,  including  five  degree-of-freedom  medium  fidelity  avionics 
and  flight  model.  The  MRH-90  cockpit  is  a  full-scale  cockpit.  The  participants  were  seated  in 
the  right  seat  of  the  cockpit,  facing  the  front  window.  The  cockpit  was  angled  at  minus 
60  degrees  to  the  normal  of  the  partial  dome  (Figure  1).  The  buoy  was  visible  out  the  right- 
hand  window,  in  accordance  with  the  configuration  of  the  DSTO  AOSC  Maritime  Hover 
research.  The  experimenters  were  seated  in  the  simulation  control  room  with  intercom 
communication  to  the  participant  in  the  cockpit. 


2.3  Procedure 

In  accordance  with  the  DSTO  AOSC  Maritime  Hover  trial,  the  buoy  and  aircraft  were 
placed  in  open  water  with  no  landforms  visible.  The  aircraft  was  initially  placed  50  m  south  of 
the  buoy.  This  50  metre  distance  was  defined  as  the  baseline  distance  Db.  Db  was  chosen  in 
accordance  with  the  DSTO  AOSC  Maritime  Hover  trial  prescribed  hover  distance. 
Throughout  the  experiment,  the  aircraft  was  moved  along  the  north-south  axis,  depicted  in 
Figure  2  by  the  dotted  line.  The  only  object  in  the  field  of  view  of  the  participant  was  the  buoy. 
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which  had  actual  dimensions  of  3.05  m  by  7.56  m.  The  simulated  buoy  was  modelled  from  a 
real  buoy,  and  had  appearance  and  dimensions  as  shown  in  Figure  2.  The  blue  wavy  line 
depicts  the  water  level. 


Figure  2  Buoy  appearance  and  dimensions 


While  the  buoy  remained  stationary,  the  position  of  the  aircraft  varied  between  0  and 
100  m  south  of  the  buoy.  Any  aircraft  position,  for  which  the  distance  between  the  aircraft  and 
buoy  varied  from  Db,  was  denoted  as  Dv.  The  positions  chosen  for  Dv  are  referred  to  as  'data 
points'.  The  differences  between  Db  and  Dv  ranged  from  0  to  50  m.  Table  1  and  Figure  4  show 
the  values  for  Db  and  Dv  at  the  chosen  data  points.  Figures  Cl  and  C2  in  Appendix  C  show  the 
same  information.  The  values  for  Dv  were  to  be  compared  to  Db  and  identified  as  'closer'  or 
'farther'. 


The  values  for  Dv  were  determined  using  the  equation  given  in  Figure  3,  for  x  values 
from  1  to  18.  This  method  was  chosen  as  it  was  assumed  that  the  furthest  distances  from  Db 
would  be  easily  identified  as  being  closer  or  farther,  and  that  the  threshold  would  lie 
somewhere  close  to  Db.  With  this  assumption,  and  by  clustering  data  points  around  Dl„  it  was 
hoped  that  the  interval  the  threshold  lies  within  would  be  narrowed.  The  values  determined 
using  this  equation  are  shown  in  Table  1.  The  distances  of  the  aircraft  from  the  origin  and 
from  the  buoy  are  shown  respectively  in  Figures  Cl  and  C2. 


50+  50 


y=  < 


2 

50+  50 


x-l 


,18-x 


1  <  x  <  10 


10  <  x  <  18 


Figure  3  Equation  for  determining  data  points 
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▲ 


100m 


▼ 


i  l 


Figure  4  Birds  eye  view  of  buoy  and  aircraft  positioning  in  Di,  and  Dv  positions 

Gescheider  (2007)  describes  the  classical  psychophysical  technique,  the  Method  of 
Constant  Stimuli,  in  which  stimuli  of  varying  strength  or  magnitude  are  evaluated  against  a 
single  constant  stimulus.  A  variation  of  the  Method  of  Constant  Stimuli  was  used  for  this 
experiment.  The  participants  were  seated  in  the  aircraft  as  shown  in  Figure  1.  An  image  of  the 
buoy  at  Db  (50  m  away)  was  projected  onto  the  simulator  screen  for  10  seconds.  The  image 
was  removed  and  replaced  with  a  grey  screen  for  two  seconds,  followed  by  a  view  of  the  first 
variation  distance,  Dvi,  also  shown  for  two  seconds.  The  participant  was  required  to  identify 
whether  Dvi  was  'closer'  or  'farther'  than  Db.  Refer  to  Figure  5  for  sample  images  of  'farther', 
baseline  and  'closer'  buoy  distances  (though  the  experiment  employed  a  realistic  buoy, 
modelled  on  the  buoy  in  Figure  2).  Once  they  had  responded,  the  grey  screen  was  shown 
again,  followed  by  Db,  then  the  grey  screen,  and  finally  DV2,  each  for  two  seconds.  Once  again 
the  participant  was  asked  to  identify  whether  DV2  was  closer  or  farther  away  than  Db.  This 
process  was  repeated  until  all  variation  distances  had  been  presented.  The  variation  distances 
were  randomised  for  presentation  for  each  participant.  The  participants  were  only  permitted 
to  respond  with  'closer'  or  'farther'. 

The  participants  were  presented  with  two  sets  of  data  points  on  different  sea  states. 
The  first  was  Sea  State  1,  characterised  by  a  smooth  and  glassy  ocean  with  little  or  no  wave 
movement.  The  second  was  Sea  State  4,  characterised  by  constant  wave  movement  of  1.25  to 
2.5  m.  Further  Sea  State  information  is  provided  in  Appendix  B.  The  Sea  State  1  condition  was 
presented  first,  followed  immediately  by  the  Sea  State  4  condition.  In  the  AOSC,  the  centre  of 
the  buoy's  base  is  fixed  to  a  position  just  below  the  surface  of  the  water.  The  rest  of  the  buoy  is 
free  to  move  with  wave  movement.  As  such,  the  buoy  tilted  with  the  motion  of  the  waves  but 
did  not  move  from  its  fixed  position. 
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Figure  5  Out-the-window  views  of  buoys  in  Dv  >  Db  (farther'),  Db  (baseline)  and  Dv  <  Db 

('closer')  positions  respectively 

This  methodology  was  chosen  to  replicate  the  cognitive  processes  of  a  pilot  executing 
flight  test  trials  in  the  simulator.  The  grey  screen  was  presented  between  image  presentations 
to  replicate  the  break  in  visual  perception  experienced  when  a  pilot  glances  away  from  the 
buoy  to  observe  his/her  instruments  or  other  elements  in  the  environment.  It  was  also 
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presented  to  prevent  correct  identification  occurring  simply  because  of  the  sudden  movement 
of  the  buoy  in  the  visual  scene  when  different  distances  were  presented. 

Table  1  Data  point  information 


Data  Point 

Dv  (m) 

Db  -  Dv  (m) 

Correct  Response 

1 

0 

50 

Closer 

2 

25 

25 

Closer 

3 

37.5 

12.5 

Closer 

4 

43.75 

6.25 

Closer 

5 

46.88 

3.13 

Closer 

6 

48.44 

1.56 

Closer 

7 

49.22 

0.78 

Closer 

8 

49.61 

0.39 

Closer 

9 

49.80 

0.20 

Closer 

10 

50.20 

-0.20 

Farther 

11 

50.39 

-0.39 

Farther 

12 

50.78 

-0.78 

Farther 

13 

51.56 

-1.56 

Farther 

14 

53.13 

-3.13 

Farther 

15 

56.25 

-6.25 

Farther 

16 

62.5 

-12.5 

Farther 

17 

75 

-25 

Farther 

18 

100 

-50 

Farther 

3.  Results 


Paired  sample  t-tests1  were  conducted  on  the  mean  response  for  each  data  point 
paired  with  the  100  %  correct  response,  to  determine  whether  the  mean  response  for  those  data 
points  was  significantly  different  from  the  correct  response.  This  was  conducted  for  both  sea 


1  The  t-test  is  a  statistical  test  used  to  identify  how  dissimilar  two  populations  are.  The  t-score  is  the 
statistic  calculated  by  performing  a  t-test.  The  p-value  indicates  how  significant  the  results  were,  or 
how  confident  we  can  be  that  the  two  populations  are  different.  The  generally  accepted  cutoff  for  a  p- 
value  is  0.05  or  less,  or  that  we  can  be  95%  confident  that  our  populations  are  different.  In  this  case,  the 
populations  were  the  participants'  responses  for  each  of  the  data  points  (each  was  a  separate 
population),  compared  with  two  fictional  populations  -  the  first  in  which  every  person  responded 
correctly  (100%  accuracy),  and  the  second  in  which  the  participants  responded  with  50%  accuracy  (the 
level  of  accuracy  we  can  assume  if  each  person  is  randomly  guessing).  The  standard  deviation  for  a 
data  point  is  represented  by  'o',  and  denotes  how  much  the  data  tend  to  vary  from  the  mean. 
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states.  Paired  sample  t-tests  were  also  conducted  to  compare  the  mean  response  for  each  data 
point  with  the  50%  correct  mark,  to  determine  the  data  point  at  which  the  mean  response  was 
no  longer  correct  more  often  than  would  be  achieved  from  guessing.  The  proportions  for  each 
data  point  of  participants  giving  'farther'  as  their  response  are  shown  in  Figure  6. 

For  Sea  State  1,  participants  responded  significantly  worse  than  perfect  for  data  points 
9  (t=-2.58,  p<0.05),  10  (t=4.24,  p<0.01),  11  (t=2.95,  p<0.01)  and  12  (t=2.95,  p<0.01)  (see  Table  2). 
For  Sea  State  1,  none  of  these  data  points  were  significantly  more  accurate  than  the  50%  mark  - 
data  points  9  (t=1.81,  p=.09),  10  (t=0.24,  p=0.82),  11  (t=-1.23,  p=0.24),  and  12  (t=-1.23,  p=0.48) 
were  all  non-significant  (see  Table  3).  Participants  showed  perfect  discrimination  for  data 
points  1,  2,  3,  4,  5, 15, 16, 17  and  18  in  Sea  State  1. 


For  Sea  State  4,  participants  responded  significantly  worse  than  perfect  for  data  points 
7  (t=-2.95,  p<0.01),  8  (t=-2.58,  p<0.05),  9  (t=-3.35,  p<0.01),  10  (t=5.42,  p<0.01),  11  (t=7.21, 
p<0.01),  12  (t=-,  p<0.05)  and  13  (t=-2.58,  p<0.05)  (see  Table  4).  For  Sea  State  4,  the  following 
data  points  were  not  significantly  more  accurate  that  the  50%  mark  -  data  point  7  (t=1.23, 
p=0.24),  data  point  8  (t=1.81,  p=0.09),  data  point  9  (t=0.72,  p=0.45),  data  point  10  (t=1.23, 
p=0.24),  data  point  12  (t=-0.72,  p=0.48)  and  data  point  13,  (t=-1.23,  p=0.24)  (see  Table  5).  Data 
point  11  was  significantly  different  from  the  50%  mark,  t=2.46,  p<0.05,  but  fell  below  the  50% 
line,  that  is,  participants  were  responding  worse  than  if  they  were  merely  guessing. 
Participants  showed  perfect  discrimination  for  data  points  1,  2,  3,  4,  16,  17  and  18  in  Sea 
State  4. 


Table  2  Paired  t-tests  comparing  Sea  State  1  data  points  against  100%  correct  mark.  Perfectly 
discriminated  data  points  are  excluded. 


t 

o 

P 

Data  Point  6 

-1.85 

0.39 

0.83 

Data  Point  7 

-1.85 

0.39 

0.83 

Data  Point  8 

-1.46 

0.33 

0.16 

Data  Point  9 

-2.58 

0.47 

<0.05 

Data  Point  10 

5.41 

0.49 

<0.01 

Data  Point  11 

2.95 

0.49 

<0.01 

Data  Point  12 

2.95 

0.49 

<0.01 

Data  Point  13 

1.46 

0.33 

0.16 

Data  Point  14 

1.00 

0.24 

0.33 
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Table  3  Paired  t-tests  comparing  Sea  State  1  data  points  against  50%  correct  mark.  Perfectly 
discriminated  data  points  are  excluded. 


t 

o 

P 

Data  Point  6 

3.40 

0.39 

<0.01 

Data  Point  7 

3.40 

0.39 

<0.01 

Data  Point  8 

4.74 

0.33 

<0.01 

Data  Point  9 

1.81 

0.47 

0.09 

Data  Point  10 

0.24 

0.51 

0.82 

Data  Point  11 

-1.23 

0.49 

0.24 

Data  Point  12 

-1.23 

0.49 

0.24 

Data  Point  13 

-4.75 

0.33 

<0.01 

Data  Point  14 

-7.5 

0.24 

<0.01 

Table  4  Paired  t-tests  comparing  Sea  State  4  data  points  against  100%  correct  mark.  Perfectly 
discriminated  data  points  are  excluded. 


t 

o 

P 

Data  Point  5 

-1.85 

0.39 

0.83 

Data  Point  6 

-1.85 

0.39 

0.83 

Data  Point  7 

-2.95 

0.49 

<0.01 

Data  Point  8 

-2.58 

0.47 

<0.05 

Data  Point  9 

-3.35 

0.51 

<0.01 

Data  Point  10 

5.41 

0.49 

<0.01 

Data  Point  11 

7.21 

0.44 

<0.01 

Data  Point  12 

3.35 

0.51 

<0.01 

Data  Point  13 

2.95 

0.49 

<0.01 

Data  Point  14 

1.00 

0.24 

0.33 

Data  Point  15 

1.00 

0.24 

0.33 

Table  5  Paired  t-tests  comparing  Sea  State  4  data  points  against  50% 
discriminated  data  points  are  excluded. 

correct  mark.  Perfectly 

t 

o 

P 

Data  Point  5 

3.40 

0.39 

<0.01 

Data  Point  6 

3.40 

0.39 

<0.01 

Data  Point  7 

1.23 

0.49 

0.24 

Data  Point  8 

1.81 

0.47 

0.09 

Data  Point  9 

0.71 

0.51 

0.48 

Data  Point  10 

1.23 

0.49 

0.24 

Data  Point  11 

2.50 

0.44 

<0.05 

Data  Point  12 

-0.71 

0.51 

0.48 

Data  Point  13 

-1.23 

0.49 

0.24 

Data  Point  14 

-7.5 

0.24 

<0.01 

Data  Point  15 

-7.5 

0.24 

<0.01 
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Figure  6  For  each  data  point,  proportion  of  participants  giving  'Farther'  as  their  response 


Paired  samples  t-tests  were  performed  for  pairs  of  data  points  that  were  equidistant 
from  Db  (for  example.  Data  points  5  and  14  are  both  3.125  m  from  baseline,  but  Data  Point  5  is 
3.125  m  closer  and  Data  Point  14  is  3.125  m  farther).  From  the  two  sea  states,  only  one  data 
point  was  significantly  different  from  its  pair.  Sea  State  4  Data  Point  11  was  significantly  less 
correctly  responded  to  than  Data  Point  8,  t  =  3.11,  p<0.01.  The  following  pairs  of  data  points 
were  no  different  from  one  another:  1  and  18;  2  and  17;  and  3  and  16.  The  results  for  the  other 
pairs  of  data  points  are  shown  in  Tables  6  and  7.  Figure  7  shows  the  percentage  of  correct 
responses  for  data  for  Sea  States  1  and  4. 


Table  6  Paired  t-tests  comparing  Sea  State  1  data  points  equidistant  from  Db 


'Closer'  Data 
Point 

'Farther'  Data 

Point 

t 

o 

P 

Data  Point  5 

Data  Point  14 

1.00 

0.24 

0.33 

Data  Point  6 

Data  Point  13 

-0.57 

0.43 

0.58 

Data  Point  7 

Data  Point  12 

1.00 

0.73 

0.33 

Data  Point  8 

Data  Point  11 

1.46 

0.66 

0.16 

Data  Point  9 

Data  Point  10 

1.17 

0.83 

0.26 
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Table  7  Paired  t-tests  comparing  Sea  State  4  data  points  equidistant  from  Db 


'Closer'  Data 
Point 

'Farther'  Data 

Point 

t 

o 

P 

Data  Point  4 

Data  Point  15 

1.00 

0.24 

0.33 

Data  Point  5 

Data  Point  14 

1.00 

0.49 

0.33 

Data  Point  6 

Data  Point  13 

1.14 

0.63 

0.27 

Data  Point  7 

Data  Point  12 

0.37 

0.66 

0.72 

Data  Point  8 

Data  Point  11 

3.11 

0.62 

<0.01 

Data  Point  9 

Data  Point  10 

1.73 

0.56 

0.10 

Figure  7  For  each  data  point,  percentage  of  correct  responses 

Paired  sample  t-tests  for  equivalent  Data  Points  from  the  two  different  sea  states  were 
performed.  The  response  for  Data  Point  11  was  significantly  worse  in  Sea  State  4  than  Sea 
State  1.  No  other  data  point  pair  was  significantly  different.  The  results  from  these  t-tests  are 
shown  in  Table  8.  Data  Points  1, 2, 3, 4,  6, 14, 16, 17  and  18  yielded  the  same  results  regardless 
of  Sea  State. 
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Table  8  Paired  t-tests  comparing  equivalent  Sea  State  1  and  Sea  State  4  data  points 


Data  Point 

t 

o 

P 

Data  Point  5 

1.85 

0.39 

0.08 

Data  Point  7 

1.38 

0.53 

0.19 

Data  Point  8 

1.38 

0.53 

0.19 

Data  Point  9 

0.70 

0.70 

0.50 

Data  Point  10 

0.70 

0.70 

0.50 

Data  Point  11 

2.75 

0.62 

<0.05 

Data  Point  12 

0.37 

0.66 

0.72 

Data  Point  13 

1.73 

0.56 

0.10 

Data  Point  15 

1.00 

0.24 

0.33 

Data  fitting  using  the  Weibull  cumulative  distribution  function  (Weibull,  1951)  was 
applied  to  the  means  data  plotted  in  Figure  6  to  find  the  underlying  psychometric  function  (s- 
curve)  to  fit  the  data.  Data  points  1  and  18  were  excluded  so  the  curve  could  be  fitted.  The 
equations  for  the  curve  fitted  to  the  data  are  given  in  Figure  8.  Figure  9  and  10  show  the  plots 
of  these  curves.  For  Sea  State  1,  the  25, 50  and  75%  cumulative  probabilities  occur  at  x  =  49.55, 
50.34  and  50.97  m  respectively.  For  Sea  State  4,  the  25,  50  and  75%  cumulative  probabilities 
occur  at  x  =  49.05,  50.77  and  52.17  m  respectively. 


-(v/50.68)55'68 

Sea  State  1:  y  =  1  -  e 


-(x/51.50)25'54 

Sea  State  4:  y  ~  1  -  G 


Figure  8  Fitted  Weibull  equations  for  Sea  State  1  and  Sea  State  4  fitted  curves 
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4.  Discussion 


The  results  for  this  experiment  indicate  four  intervals  in  which  thresholds  for  distance 
discrimination  lie  in  the  AOSC  flight  simulator.  For  each  sea  state,  there  is  one  threshold  for 
discriminating  distances  closer  than  the  50  m  baseline  distance,  and  one  for  distances  farther 
than  baseline.  When  determined  for  Sea  States  1  and  4,  these  thresholds  may  be  compared  to 
the  'desirable'  and  'adequate'  performance  limits  of  the  maritime  hover  task.  It  may  then  be 
shown  whether  the  distances  the  limits  prescribe  are,  in  fact,  too  small  to  be  reliably  perceived 
in  the  simulator. 


4.1  Thresholds 

4.1.1  Sea  State  1 

For  Sea  State  1,  for  conditions  in  which  the  aircraft  is  closer  to  the  buoy  than  Db,  the 
threshold  for  distance  discrimination  in  the  AOSC  flight  simulator  appears  be  0.45  m.  That  is, 
the  aircraft  must  move  at  least  0.45  m  towards  the  buoy  for  that  movement  to  be  identified  by 
the  pilot.  As  3  ft,  the  limit  for  'desirable'  amounts  of  aircraft  movement,  is  equal  to  0.91  m  and 
is  larger  than  this  threshold,  a  pilot  can  be  reasonably  expected  to  perceive  a  movement  of  3  ft 
towards  the  buoy  in  Sea  State  1. 

For  distances  farther  from  the  buoy  than  Db,  the  threshold  for  distance  discrimination 
for  movement  away  from  the  buoy  in  the  AOSC  flight  simulator  appears  be  0.97  m.  The 
aircraft  must  move  a  distance  of  at  least  0.97  m  away  from  the  buoy  for  that  change  in  distance 
to  be  correctly  identified.  The  3  ft  'desirable'  performance  limit  is  less  than  this  threshold, 
hence  it  cannot  be  expected  that  a  pilot  identify  a  3  ft  movement  away  from  the  buoy.  A  pilot 
should  be  able  to  identify  a  movement  of  the  'adequate'  distance  of  6  ft  (1.83  m)  in  Sea  State  1. 

4.1.2  Sea  State  4 

For  Sea  State  4,  for  conditions  in  which  the  aircraft  is  closer  to  the  buoy  than  Db,  the 
threshold  for  distance  discrimination  in  the  AOSC  flight  simulator  appears  to  be  0.95  m.  The 
aircraft  must  move  at  least  0.95  m  towards  the  buoy  for  that  movement  to  be  identified  by  the 
pilot.  The  'desirable'  performance  limit  of  3  ft  (0.91  m)  is  less  than  this  threshold,  hence  it 
cannot  be  expected  that  a  pilot  identify  a  3  ft  movement  towards  the  buoy  in  the  experimental 
conditions.  A  pilot  should,  however,  be  able  to  perceive  a  movement  of  6  ft  (1.83  m,  the 
'adequate'  distance)  towards  the  buoy. 

For  distances  farther  from  the  buoy  than  Db  in  Sea  State  4,  the  threshold  for  distance 
change  identification  in  the  AOSC  flight  simulator  appears  to  be  2.17  m.  The  aircraft  must 
move  away  from  the  buoy  at  least  2.17  for  that  change  in  distance  to  be  correctly  identified. 
The  3  ft  'desirable'  performance  limit  is  less  than  this  threshold,  hence  it  cannot  be  expected 
that  a  pilot  identify  a  3  ft  movement  away  from  the  buoy.  The  'adequate'  performance 
requirement  of  6  ft  (1.83  m)  is  also  smaller  than  the  threshold,  so  we  also  cannot  assume  that 
someone  will  be  able  to  correctly  identify  a  movement  of  6  ft  away  from  the  buoy. 
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During  the  experiment,  it  became  apparent  that  the  thresholds  for  distance 
discrimination  were  not  mirrored  about  Data  Points  9  and  10  (the  centre  data  points).  For  Sea 
State  1,  as  stated  above,  for  closer  distances  the  threshold  is  0.45  m.  For  farther  distances,  it  is 
0.97  m.  A  similar  bias  is  apparent  for  Sea  State  4.  For  closer  distances,  the  threshold  is  0.95  m 
and  for  farther  distances  it  is  2.17  m.  These  results  suggest  that  it  is  more  difficult  to 
discriminate  distance  when  the  distance  between  the  participant  and  the  target  is  increasing, 
and  are  in  accordance  with  pilot  feedback  stating  that  it  is  easier  to  identify  drift  towards  the 
buoy  than  away  from  it.  However,  comparing  data  points  that  were  equidistant  from  Db 
showed  no  significant  difference,  except  for  Sea  State  4  data  points  8  and  11.  Thus,  although 
the  magnitudes  of  the  thresholds  appear  to  indicate  a  trend  towards  poorer  performance 
when  judging  further  distances,  this  is  not  explicitly  shown  by  the  data.  Further  testing  with  a 
larger  sample  size  may  reveal  a  significant  difference  in  performance  for  'closer'  and  'farther' 
distances. 

Pilots  indicated  after  preliminary  testing  for  the  DSTO  AOSC  Maritime  Hover  trial 
that  it  was  easier  to  maintain  plan  position  in  the  lower  sea  state  as  they  were  able  to  use  static 
environmental  elements  as  cues  to  their  position  relative  to  the  buoy.  Examples  of 
environmental  elements  that  were  mentioned  included  the  reflection  of  the  clouds  on  the 
water,  and  the  position  of  the  buoy  relative  to  the  line  of  the  horizon.  When  there  was 
considerable  wave  movement,  the  pilots  found  it  very  difficult  to  use  these  same  cues  to  judge 
the  distance.  Pilots  reported  that  the  Sea  State  1  condition  presented,  in  essence,  a  static 
environment  in  which  visual  cues  could  be  used  to  judge  distance  as  they  would  in  a  static 
land  environment.  However,  as  above,  the  results  of  this  experiment  do  not  reinforce  pilot 
feedback.  No  significant  degradation  in  performance  was  found  for  any  data  point  except 
Data  Point  11.  Nevertheless,  observing  Figure  6,  the  trend  once  again  appears  to  be  towards 
poorer  performance  judging  distance  in  a  higher  sea  state.  Once  again,  testing  a  larger  number 
of  participants  in  future  work  may  reveal  a  significant  difference. 

Pilot  reports  indicating  greater  ease  in  maintaining  position  in  the  AOSC  Sea  State  1 
environment  is  contrary  to  feedback  from  actual  flight  trials,  which  suggests  that  that  Sea 
State  1  actually  provides  the  fewest  visual  cues  and  is  the  hardest  sea  state  in  which  to 
maintain  plan  position.  In  higher  Sea  States,  wave  motion  and  visible  currents  can  actually  be 
of  assistance  when  judging  distance.  In  the  simulator  this  outcome  is  not  reproduced  -  Sea 
State  1  appeared  to  have  smaller  thresholds  for  distance  discrimination,  both  for  closer  and 
farther  distances,  even  if  this  difference  was  not  significant.  Examining  the  available  cues  may 
give  some  indication  as  to  why  this  might  be.  Cloud  reflections  -  though  present  in  actual 
flight  test  environments  -  are  not  as  reliably  static  in  an  actual  maritime  environment  as  in  the 
simulated  maritime  environment.  In  the  simulated  environment  they  are  a  false  cue, 
highlighting  a  problem  with  performing  the  task  in  the  AOSC  flight  simulator  -  if  cues  that 
are  readily  used  in  actual  flight  trials  (such  as  rotor  downwash  and  spume  patterns)  are  not 
present  in  the  simulated  environment,  and  other  cues  that  do  not  exist  in  flight  trials  are  used 
instead,  pilots  must  be  completing  the  task  using  some  alternative  method.  The  implications 
of  this  must  be  considered.  The  ecological  validity  of  the  maritime  hover  study  -  that  is,  the 
ability  to  compare  the  AOSC  simulator  findings  to  pilots  completing  the  task  in  actual  flight 
trials  (Shadish,  Cook  &  Campbell,  2002)  -  may  be  compromised  if  cues  comparable  to  these 
flight  trials  are  not  provided. 
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4.2  Implications 

In  Sea  State  1,  for  farther  distances,  and  in  Sea  State  4,  for  both  closer  and  farther 
distances,  a  pilot  will  not  be  able  to  perceive  a  movement  of  3  ft.  If  a  pilot  is  required  to 
maintain  plan  position  within  a  limit  that  is  too  small  to  be  detected  at  a  perceptual  level,  it 
will  only  be  by  chance  than  the  hover  is  successfully  completed.  This  is  not  adequate 
assurance  for  testing  a  potential  manoeuvre  in  the  simulator. 

In  those  cases  where  the  threshold  is  smaller  than,  but  very  close  to,  the  3  ft  'desirable' 
performance  limit,  other  cognitive  processes,  active  during  aircraft  operation,  may  interfere 
with  perception.  If  the  limit  is  only  0.46  m  from  the  perceptual  threshold,  as  is  the  case  for  Sea 
State  1  closer  distances,  any  additional  workload  would  only  have  to  interfere  with  task 
performance  to  the  extent  that  an  additional  46  cm  (0.92%  of  the  total  distance  from  the  buoy) 
goes  unnoticed  for  the  'desirable'  limit  to  become  undetectable.  Performing  a  maritime  hover, 
which  requires  aircraft  operation  whilst  maintaining  altitude  and  heading,  could  provide 
sufficient  workload  to  degrade  performance.  Furthermore,  task  complexity  for  the  maritime 
hover  may  be  increased  by  the  missing  or  false  visual  cues  in  a  flight  simulator.  If  a  pilot  is 
required  to  maintain  plan  position  within  a  limit  that  is  very  close  to  the  threshold  for  distance 
discrimination,  the  likelihood  that  the  hover  will  be  successfully  completed  will  also  be  low, 
given  the  complexity  of  the  full  hover  task. 


4.3  Caveats  and  Future  Work 

Presenting  the  baseline  distance  only  once,  instead  of  between  each  data  point,  could 
extend  the  study  to  increase  external  validity.  In  actual  flight  trials,  pilots  will  only  view  their 
initial  distance  momentarily  before  having  to  compare  it  to  their  subsequent  two  minutes  of 
hover  position.  Slight  amounts  of  drift  that  will  inevitably  occur  may  not  be  registered,  and 
hence,  for  the  remainder  of  the  two  minutes,  the  pilot  may  attempt  to  maintain  a  position  that 
is  offset  by  a  significant  error  that  compounds  for  the  duration  of  the  trial.  It  is  suspected  that 
this  would  increase  task  difficulty,  and  therefore  increase  the  threshold  for  distance 
discrimination.  Alternatively,  the  buoy  could  be  presented  continuously  drifting,  allowing  the 
pilot  utilise  to  the  usual  dynamic  drift  cues  experienced  when  fixating  on  the  buoy.  It  is 
suspected  that  this  would  also  reveal  a  different  threshold. 

Unfortunately,  it  was  not  feasible  to  use  only  Navy  pilots  as  participants  for  this 
experiment.  Flaving  had  the  training  and  experience  of  using  subtle  environmental  cues  to 
judge  their  position,  it  is  expected  that  Navy  pilots  would  have  a  greater  ability  to  judge 
distance  in  a  dynamic  maritime  environment.  However,  the  cues  in  the  AOSC  dynamic 
simulated  maritime  environment  are  limited.  The  fine  visual  cues  that  might  place  a  pilot  at 
an  advantage  when  judging  distance  are  simply  not  present.  Without  these  cues,  the 
advantage  a  pilot  should  enjoy  does  not  exist.  Though  a  pilot  may  have  a  greater  ability  to 
judge  distance  in  the  presence  of  subtle  environmental  cues,  without  such  cues,  the  task 
becomes  merely  perceptual  instead  of  skill-based. 

Ultimately,  even  in  a  task  where  workload  is  minimised,  perceptual  thresholds  for 
distance  discrimination  lie  close  to  performance  limits.  Allowing  for  some  human  error  and 
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accounting  for  switching  attention  as  other  aspects  of  the  maritime  hover  are  attended  to,  it 
becomes  increasingly  likely  that  the  pilot  will  drift  outside  of  the  3  or  6  ft  radius  they  are 
required  stay  within.  At  its  current  stage,  the  simulator  does  not  appear  to  be  an  appropriate 
environment  in  which  to  complete  a  low  altitude  maritime  hover,  with  reference  to  a  buoy 
50  m  away.  If  it  is  to  be  successfully  completed  in  the  AOSC  simulator,  some  aspect  of  the 
simulator  or  the  task  must  be  altered. 


4.4  Suggested  Solutions 

To  rectify  the  problems  pilots  experience  when  performing  the  hover  in  the  simulator, 
3  solutions  may  be  employed.  These  are  as  follows: 

•  The  first  is  to  improve  simulator  technology  to  include  more  of  the  fine  visual  cues  a 
pilot  expects,  such  as  the  rotor  spray  and  spume  patterns  the  aircraft  makes  in  the 
water.  Simulator  improvement  is  a  continuing  and  cumulative  process,  so  this  will 
not  be  the  most  rapid  solution. 

•  The  second  is  to  specify  how  far  from  the  buoy  the  pilot  should  hover.  A  3  ft 
movement  backwards  or  forwards  will  change  the  image  projected  onto  the  screen  by 
a  larger  proportion  at  closer  distances,  decreasing  the  threshold  for  discrimination. 
Hence,  a  prescribed  distance  would  have  to  be  less  than  the  50  m  used  for  this 
experiment.  Further  testing  will  have  to  be  performed  to  find  the  optimal  distance  if 
this  solution  is  to  be  employed. 

•  The  third  option  is  to  review  the  maritime  hover  performance  limits  (for  plan  position 
in  particular)  and  widen  them  to  account  for  the  degraded  performance  observed  in 
the  simulator. 

Of  course,  a  combination  of  these  options  may  prove  to  be  the  most  effective  solution. 


5.  Conclusion 


Although  additional  testing  is  required  to  narrow  further  the  region  in  which  the 
distance  discrimination  threshold  lies,  the  results  of  this  experiment  suggest  that  the  threshold 
is  situated  very  close  to  or  beyond  the  3  ft  'desirable'  limit  of  the  maritime  hover  task,  and  in 
some  cases,  close  to  or  beyond  the  6  ft  'adequate'  performance  limit.  This  is  true  in  the  absence 
of  any  task  other  than  identifying  distance.  It  is  therefore  unreasonable  to  expect  pilots  to 
maintain  the  current  'desirable'  and  sometimes  'adequate'  distances.  While  simulator 
improvement  continues,  a  review  of  these  limits,  or  alternatively  a  prescription  for  the  precise 
distance  to  hover  from  the  buoy,  should  be  considered.  This  will  ensure  that  limits  for  the 
maritime  hover  are  a  true  reflection  of  the  aircraft's  capacity  to  perform  the  manoeuvre  and 
that  such  a  capacity  is  not  impeded  by  a  pilot's  inability  to  perceive  small  distance  changes  in 
the  AOSC  simulated  maritime  environment. 
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Appendix  A:  Maritime  Hover  Performance  Criteria 

Taken  from  DSTO-TN-0936 


A.l.  Proposed  Maritime  Hover  Mission  Task  Elements 

A.  1.1  Objective 

Check  ability  to  maintain  precise  position,  heading,  and  altitude  in  the  presence  of  calm 
winds  and  moderate  winds  from  the  most  critical  direction. 

A.  1.2  Description  of  manoeuvre 

Establish  and  maintain  hover  over  the  target  point.  For  moderate  wind,  orient  the  aircraft 
with  wind  at  the  most  critical  azimuth. 

A.1.3  Description  of  test  course 

Over  water  the  manoeuvre  should  be  flown  to  a  fixed  buoy  with  only  open  water  visual 
references  available.  For  baselining,  the  manoeuvre  should  be  flown  to  an  appropriate 
land-based  target  point. 


A.2.  Performance  Criteria 

Table  A1  Maritime  Hover  performance  criteria 


Criteria 

Desired 

Adequate 

Maintain  plan  position  within  _X  ft  of  the  target  point 

3  ft 

6  ft 

Maintain  altitude  within  _X  ft 

4  ft 

6  ft 

Maintain  heading  within  _X  deg 

5  deg 

10  deg 

Maintain  hover  for  _X  min 

2 

2 
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Figure  A1  Plan  position  performance  limits.  The  central  black  circle  represents  the  'target  point'  or 
point  of  origin  of  the  hover.  The  inner  circle  represents  the '  desirable'  performance  limit;  the 
outer  circle  represents  the ' adequate' performance  limit.  A  pilot  must  stay  within  the  inner 
circle  to  maintain  ‘  desirable'  performance  for  plan  position.  If  they  breach  the  inner  circle 
but  stay  within  the  outer  circle,  they  will  perform  within  the  'adequate'  limits  for  plan 
position. 


22 


UNCLASSIFIED 


UNCLASSIFIED 


DSTO-TR-2701 


Appendix  B:  Sea  State  Information 


Table  B1  Sea  state  information 


Sea  State  Code 

Wave  Lleight  (m) 

Characteristics 

0 

0 

Calm  (glassy) 

1 

0  to  0.1 

Calm  (rippled) 

2 

0.1  to  0.5 

Smooth  (wavelets) 

3 

0.5  to  1.25 

Slight 

4 

1.25  to  2.5 

Moderate 

5 

2.5  to  4 

Rough 

6 

4  to  6 

Very  rough 

7 

6  to  9 

High 

8 

9  to  14 

Very  high 

9 

Over  14 

Phenomenal 
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19.  ABSTRACT 

The  Aeronau  tical  Design  Standard  33  (ADS-33)  prescribes  a  set  of  manoeuvres  and  limits  for  assessing  the  handling  qualities  of  military 
rotorcraft.  In  developing  an  ADS-33  maritime  addendum,  the  Aircraft  Maintenance  and  Flight  Trial  Unit  (AMAFTU)  and  Defence 
Science  and  Technology  Organisation  (DSTO)  have  collaboratively  worked  towards  the  definition  for  a  maritime  hover  manoeuvre. 
Initial  testing  to  validate  this  proposed  manoeuvre  in  the  DSTO  Air  Operations  Simulation  Centre  (AOSC)  flight  simulator  has  proved 
problematic.  Although  pilots  could  successfully  complete  a  land-based  hover  within  performance  limits  in  the  flight  simulator,  they 
experienced  difficulty  in  maritime  conditions.  Pilots  were  unable  to  perceive  small  distance  changes  in  the  AOSC  simulated  maritime 
environment  in  some  conditions. 
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